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_Summary—The problem of the optimum detection of signals in 
ise, when a possible signal can arise from more than one class of 

al types, is considered from the point of view of decision theory. 
pecifically, optimization here consists of minimizing the average 
sk for preassigned costs appropriate to the possible correct and 
correct decisions, when only a single signal, of class k, may occur 
it of N + 1 mutually exclusive signal classes. The analysis is a 
neralization of the authors’ earlier work! on binary, or single 
ie detection systems. The present treatment outlines the 
timization procedure for additive signals and noise, indicates the 
mneral structure of the detector, and gives expressions for the 
jobabilities of error and minimum average (or Bayes) risk. Some 
ere results for normal statistics are included, and an example— 


yherent detection of similar signals, differing only in amplitude— 
strates the general approach. 


INTRODUCTION 


“ “\ ONSIDERABLE attention has been given recently 
Lz to optimum detection systems for determining the 
presence or absence of a signal in noise’'’* *, when 
ily a single alternative is involved; i.e., when two decisions 
aly are made: “‘yes,”’ a signal (as well as noise) is present, 
« “no,” only noise occurs. As in these simple binary 
pses, the methods of decision theory’''* are also particu- 
rly well-suited to a systematic treatment of multiple 
ternative situations involving the possible presence of 
ore than one signal, or of a signal from more than one 
ta or class of signals, during a typical observation 
iterval (0, 7). 

Accordingly, the purpose of this paper is to extend the 
inary theory to include those multiple alternative situa- 
ions where a single signal only can appear (with noise) 
1 any one observation, but the signal itself may be 
i eh from a large number of different types or classes, 


| * A portion of this work was carried out by D. Middleton, as a 
insultant to Melpar, Inc., Cambridge 39, Mass., as part of a study 
onsored by the Department of Defense under Contract DA-49- 
(0-sc-1391. Permission for publication is gratefully acknowledged. 
+49 Lexington Ave., Cambridge 38, Mass. 
-{Cruft Laboratory, Harvard University, Cambridge 38, Mass.; 
»w at Melpar, Inc., Cambridge 39, Mass. i? 
fr D. Van Meter and D. Middleton, ‘““Modern statistical approaches 
| reception in. communication theory,’ Trans. IRE, vol. IT-4, 
119; September, 1954. 
la), Middleton and D. Van Meter, ‘‘Detection and extraction of 
nals in noise from the point of view of statistical decision theory,” 
be published in Jour. Soc. Indus. & Appl. Math. (SIAM), vol. 3, 
.4, December, 1955, and vol. 4, no. 1, March, 1956. 

21). Middleton, ‘‘Statistical detection of pulsed carriers in noise. 
Il,” Jour. Appl. Phys., vol. 24, pp. 371, 379; 1953—also Trans. 
RE, vol. IT-3, p. 26; March, 1954—also Rep. No. 35, Lincoln Lab., 
.T.; November 1, 1953. For further references see above, and 
liography in references 3 and 1. 

3W. W. Peterson, T. G. Birdsall, and W. C. Fox, “The theory of 
nal detectability,’ Trans. IRE, vol. IT-4, p. 171; September, 
54—also Tech. Reps. No. 13 and 19, Electronic Defense Group, 
niversity of Michigan; July, September, 1953. 

4A. Wald, “Statistical Decision Functions,” John Wiley & Sons, 
e., New York, N. Y.; 1950. 


On Optimum Multiple-Alternative Detection 
of Signals in Noise’ 
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where the distinguishing features of each class are not 
shared by any other. For example, waveform may be 
used to distinguish between signal classes: one may 
consist of an ensemble of sinewaves; another, of square 
waves; a third, of pulses, etc. Or amplitude may be used: 
one class may be composed of sinusoids whose amplitudes 
fall between a and 2a; a second, of sinewaves with ampli- 
tudes in the range (2a, 3a) and so on. Our problem is thus 
to decide, in some optimum fashion, which of the possible 
signals is present, when the true signal in each instance is 
corrupted by noise. 

To do this, we employ here the notion of risk,’ which 
associates certain costs with each possible decision, and 
uses the expected cost (or average risk) to evaluate system 
performance. Optimum detection, from this point of view, 
is then defined as that process which minimizes the average 
risk, and this minimum average, in turn, is called the 
Bayes risk’. Our procedure is first to construct the average 
risk function, and next to minimize it by suitable choice 
of the decision operation. The result reveals the explicit 
structure of the optimum detector itself, which, as in the 
single-alternative cases, is determined by the a priori 
noise and signal statistics, signal structure, and the 
preassigned costs. 


THe AVERAGE RISK 
We begin with the average risk R(c, 6), which is defined 


1 
as 


Re, r= [av [as [ a C(S, FV | S)o(S)4(y | V). 
(1) 


Here a(S) is the a priori distribution of all possible signals 
S, divided into N + 1 distinct classes. Writing 


Cy ak S belongs to class | 
=) 


(2) 


otherwise 


we may express the distribution a(S) as 


Ta @ 


where p, are the a priori probabilities that a signal from a 
class k(=0, 1, ---, N) be present on any one observation, 
and the w, are the probability (densities) governing the 
signals that fall within the class or type k. Here k = 0 
represents the class of null signals, or noise alone. In the 
following, the superscript notation refers to the type of 


o(S) = 2 Prd” (S)wi(), 


5 For a detailed discussion of the method in communications prob- 
lems, see sections 1 and 2 of reference 1. 
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signal, or hypothesis class, while subscripts refer to a 
particular signal, or components of that signal, within a 
class. Thus Sj” = [S{?, --- Si] denotes the jth signal 
in the kth class with its n components, each of which 
represents a value of the signal at a certain time, 7.¢., 
Si? = S5(t,), ete., and these times are ordered, with 
t, <t < --- < t,. We shall use S“ to denote an un- 
specified signal of class k. 

The received data are denoted by V = [Vi, ---, Val, 
and the N + 1 hypothesis classes are taken to be mutually 
exclusive, so that the signals S$” are necessarily different 
from class to class (for different k’s) as well as within each 
class (for different 7’s). The manner in which the various 
signals and the noise are combined determines the precise 
structure of the conditional probability F(V | S) for 
the received wave V, when S is given. The quantities 
C and 6 are, respectively, the cost function and the decision 
rule (expressed as a probability density of a decision 
vy = (Yo, %1, °° *, Yn), based on the data V. Here the opera- 
tion {, over decision space A in (1) is explicitly a sum 
over the N + 1 decision points k = 0, 1, ---, N. For 
example, y, is the decision that some signal of class k, as 
well as noise, is present. Making such a decision is equiva- 
lent to testing (and accepting) the hypothesis H;, against 
all other alternative hypothesis H,;(7 4 k;7,k = 0, --- N). 
We require also that 


N 


De b,| V) = 1, (4) 


which is simply a statement of the fact that a definite 
decision of some kind must be made. 

Constant costs are next assigned to the possible out- 
comes, according to the usual procedure in risk theory’. 
We write 


cC\” = cost associated with the decision that some 
signal of class j is present, when actually a signal (and 
noise) from class k occurs, (7, k = 0, 1, --- N). 


For “‘suecessful’’ or correct decisions we have then: 


Gr 
0 . 
Gee 3 
nae 


CLS aye) noise alone correctly detected 


CS d Vi) 


signal S“”’ correctly detected (5) 
(k —— ik ae ee 


I 


The costs preassigned to “failures,” or incorrect decisions, 
are represented by 


C(S; y;) = Cf: signal S‘” incorrectly decided 

upon when only noise is pre- 

sent (7 ¥ 0) r 6) 
c(s : y;) = CM: signal S” detected, when S“’ 

is actually present (7 # k). J 


Note that in assigning costs, no distinctions between 
different signals of the same class are made. From the 
definition of failure and success we require also that 


Ce a CG = k); Ay x Cag x 0); On = 0. (7) 


SeptemBil 


In what follows we assume that signals and noise are’ 
additive and independent: V = S“’ + N; (k = 0, --- N), 
so that F(V |S) becomes W,(V — S), where W,, is the nthy 
order probability density of noise alone. Applying (2)-(7/) 
to the average risk, and writing po = q, we readily obtaal 
for (1) on integrating over decision space A, 


Ry(o, 6) = iL i 0 (Yo | V) ain = Cr.” 5(Vr iv) Jarr.cw ; 


| 
y 


+ be D4 t 8(yx | V) | 


N f 
+ a C7? 5(y; wary ss s), |} dv, (8)! 
} 


subject to the conditions (4) and (7), where (_ ), indi 
cates the statistical average over S“’, with respect a 
w,, ef. (3), and the prime on the summation means that 
(j # k). Note that where N = 1 (the binary case), (8) 
reduces with the help of (7) to the earlier aan 
(21, 22)' with obvious changes in notation, cf. (15)! 
therein, etc. dl 


) 
MINIMIZATION OF THE AVERAGE RIsK; | 
Tue Oprimum DerrectTor 


It is now convenient to rearrange (8), with the help of 
(4), by collecting coefficients of 6(y, | V). Introducing the 


quantities | 
( 


= CO = COO ie Bae) = een 


with . >107 bi 0 Ae oOo 
(a) 4 Re = a a Co (l = i eid NOS “ 
P ZSokezd| 


: 


(k k k 
i 


(Gi) AV) = pWCV — S®))./gW.(V) > 0, (10) 
(ii) AV) = + a Me AL(V), i) 


we find after some algebra that the average risk (8) may 
be written | 
Ry(o, ) = Ro ++ Ry, (12). 


wheres) =.qC)” -E Dance GC. ewe simply the; 
expected cost of calling every signal (including noise), 
‘noise,’ and 


| 


Gy = [ | > ays AW) [aac av, . (ae 


which is the part of the average risk that may be adjusted: 
through choice of 6; Ry > Ry, from the conditions ©) | 
The quantities A, (V), (k = 1, --- , N), are generalized. 
likelihood ratios,''* whose precise form determines the 
structure of our (V + 1)-ary detector. The process of. 
detection itself is now specified by our choice of the 6’s: 
and the corresponding regions for the A, (V), (all V), 
which are nonoverlapping for the mutually exclusive’ 


hypotheses considered here. , 


)Since we are interested in optimum detection, our 
oblem now is to choose 6(y, | V), for given V, in such a 
y that the average risk ®y (and hence Ry) is minimized. 
asically, this is the problem of determining the boun- 
ries of the critical regions for A, (V), which from (11) 

ad (13) we suspect will be set by a series of linear equa- 

ons in the various likelihood ratios. 

The argument for minimization is easily given: since 

WV, (V) = 0 everywhere in I, the average risk (12, 13) is 

st where for each value of V we choose 6 appropriately 

| minimize )*Y 6, A,, subject for the moment to the 

jsumption that for this given V there exists an A, (V) 

hich is (algebraically) less than all other A; (V), (j ¥ k). 

he procedure, thus, is to examine all A’s for the given V, 

llecting that one (A,) which is algebraically least, and 

ten choosing (for the same V) 6(y, | V) = 1, 6(y;| V) = 0, 

~ k, all 7). Repeating for all V in T we obtain finally a 

it of conditions in the A,’s, for all (k = 1, --- N). Note 

fom the form of (13), (the 6’s appear linearly) and from 
4e process of minimization itself, that 6 is automatically 
nonrandomized decision rule (i.e., 6 = 1 or 0 only). 
ince the A, (V) possibly contain negative parts (9), we 
ie that to minimize the average risk and make a decision 

' (signal S“° present in noise) on the basis of received 

uta V, the specific conditions on the A, must be that the 

eta V satisfy the inequalities: 


OL Ss OV Nee 2G Nae Sa (V), 

| ali hee lee scan) (14) 
le + = AAV) < 0, 

ith OU Var ev eas dis Oyen) = 


or each possible decision y,, (k = 1 --- N), this applies. 
he only remaining set of inequalities covers the case of 
pise alone (k = 0); we have here the conditions 


ie 
ee (Vy 0) - , N) 
l=1 


| 1 
ith (16) 


NEN els OV UIRV) = iOpe es CG = 


If now we regard the A’s as our independent variables, 
e can at once give a direct geometric interpretation of the 
utually exclusive sets of CORONER: (14), (15). Writing 
i) as the value of the quantity 4,” + Do7L. de? Ar (V) 
id L, for the hypersurface® L, = 0, we eee that in 
njunction with the hypersurfaces forming the boundaries 
the first “2"**-tant’’’, the equalities in the conditions 
4) or (15) give the boundaries of a closed region within 
hich lie all values of A, (V) associated with the decision 


END) 


| 


6For N > 4, Lx = 0 is a plane hypersurface (in Ai, *** , Aw- 
ace); for N = ihe 2, L; = Orepresents a straight line in 2 dimensions 
f Ay, Ao-space), while for NV = 3, Li, = O represents a plane surface 
1 Ai, As, As-space). 

7¥or example, if N = 1 the first ‘2?-tant’’ = first ‘“quadrant”’ ; 
Ve = 2, = first ‘octant, ” etc. Since likelihood ratios A; can never 
negative, values of Az(V) must always lie in the first ‘(2 +!-tant.” 
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7x. Each closed region is distinct from every other, and the 
N planar hypersurfaces which form its boundaries are 
then from (14) and (15) specified by 


ie ae ee all j =1,---N; (G#h; 


(for each y,, & = 1, -+-N) 
lw =i e yl ey cae Ns (for Yo). (17) 


Solving the N linear equations (14), (15), or (16, 17), one 
can show in straightforward fashion that the various 
(distinct) N planar hypersurfaces determining the boun- 
daries of each region all intersect at a point K = (K$” 

- K;””), where now the K’s represent a set of N thresholds 
A, (V’) = Ko”, (k = 1, --- N), in which VW’ are all values 
of V auerane this relation. These thresholds depend 
explicitly only on the preassigned costs, i.e. on the )’s of 
(14) ete., and the requirement (7) insures that the point 
K lies in the first 2”*?-tant, ie., all K™ > 0. 

A simpler variant of (14), also of practical interest, 
arises where the problem becomes that of testing for the 
presence of one signal S“” in noise against (any one of the) 
other possible nonzero signals in nozse; the case of noise 
alone is eliminated. Under these circumstances the costs 
Co One Ce drop out, and the )’s of (9) et seq., are 
pape cme OL wl b= 1, 22g OeConsequentiv: 
the A, (v) can never be ae and minimization of the 
average risk gives only the first set of inequalities in (14), 
with \,” = 5° = 0, so that we may write then for the 
decision y,, the modified conditions 


(16) 


N N 
Sei VIC De AoC ae 
l=1 a 


(j ~k;allj = - N), (18) 


repeated for each (kK = 1, --- , N) in turn. The point 
K = 0 now, and all decision regions (for the A’s) have 
their apices at the origin. The bounding hyperplanes all 
intersect at K = 0, and the equations of the boundaries of 
the kth region are simply 


Lie; = 0; allies BOT Glas 1 


(for each y,,k = 1--- N). (19) 


To summarize, then, we see that the optimum (N + 1)- 
ary (or N-ary above) detector consists of a computer 
which evaluates the A, (V) for a given set of data V over 
the observation period (0, 7), computes the various 
A, (V) and then inserts the results into the inequalities 
(14), (15), or (18), finally making the decision yj) as- 
sociated with the one set of inequalities that is satisfied." 

We illustrate the remarks of this section by: 


(a) Binary Detection 


The simplest and most familiar’ * case of (14), (15) 
arises when we have to distinguish S“’ + N versus N 


8 Of course, this is not a unique way of setting up an actual com- 
puting scheme; one might use, for example, an appropriate number of 
binary decisions with appropriate limits; but in any case, the opera- 
tion must be equivalent to the above. 
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alone, so that N = 1. With the help of (9) in (14), (15) 
we easily find that we 


decide y,: N, if A(V) <| 
decide y,: SY +4N, if A,(V) > | 


(0) (0) 
C; ena C CE re. Ca = 
Ce iC Ce, 


(0) 
Ay 

(1) 
Mi 


esas (20) 
with the threshold K;” a function of the costs only. 
[C,, Cy, ete. are expressed in the earlier notation of 
references | and 2.] 

The more general binary problem of distinguishing 
S“ + WN against S‘” + N, with a, b any single integers 
in the range (1 < a,b < N), (a ¥ b), N = 2, is readily 
treated. From (18) we write for the decision process 


A,(V) > pea! 
RAV AV 


1 Gr? ia Ge 

where Ki” = —* : 
(C0) Co 
b / a 


S@ +N, if 
SiN cit 


decide y,: 


decide y,: 


(21) 


eal), 


which can also be expressed alternatively in terms of a 
single likelihood ratio 


KV) = PAW = 8 ))s/ Pa WalV = Sane mate) 


or its reciprocal. [Typical decision regions are shown in 

Fig. 1, next column, if we replace A, by A,, A, by A, and 

allow the line L, — L, = L, — L, = 0 to go through 
(6)\ +1 


zero, with slope (K,’’) .] 


(b) Ternary Detection 


In this first example we assume that noise alone is one 
of the three possible alternatives, so that N = 2. Then 
from (14) and (15) the decision process is found at once 


7 (2) 


(k = 1, 2;1 = 1, 2). The two thresholds Ky”, Ky” are 
(0), (2) (O)\ (2) 
= ON 
q) 2 Ay iL ROD. 
K® = ; 
yD (23) 
>(2) _ Ai Ad ND 
I A 3 
es AP) = KONGs 


and we decide 
vo: N, when AX” + AY? Ar + Ai? Ae > O } 24) 


RUG Ns pe Ns Nabi Na Ay OF 


12 SY +N, when \X” + AYA $ AY? Ad 
SN uictash Age enous Na Meee) 
and Ne ate Aye 
218 + N, when AS” + AQ? AW + Ag Ae 
a ROE Noe ae (26) 


and hoo) A a ee 


The boundaries of the various decision regions are easily) 
determined from (16, 17) and the Aj,-, A».-axes bound-| 
ing the first quadrant. A typical case is illustrated in Fig. 1. 


s? +N, 8% + Nos OSCUN Gi 0 01430) (ie 


September 


The decision process is particularly simple when the 
case of noise alone is removed, and one of three possible 
combinations of signal and noise can now occur; ie) 


c < N), N = 3. From (18) we find the decision process3 


decide y,: S‘° + N, when A, > AK Sea 4 (20) 


ANS "Apher Apes 


decide y,: 8° + N, according to (27), replacing a by 6,’ 
and b by a therein. (28) 


decide y,: S‘“’ + N, according to (27), letting a > ¢, 


b > a,c — b therein, (29) 
where the thresholds K‘;” etc., are specifically 
(b) (b) (e) (ec) 
K® = Ca a Cy ‘, KS mae Cr a o . 
ceo) eae Go? KG Go? 
b 4D; 
a a (30) 
(b) (b) (c) (e) 
K® Ge ina Cc. 5 K® a Ce ni Cx F 
ae (a) (a) ? CS VA (a) 
Coney Cane Oe 


Fig. 1—Regions of decision for ternary detection (NV = 2): 
N vs. SY + N vs. S® + N.Ai® > 0; > < 0.) f 


[The thresholds for y,, y. are found by interchanging a,: 
b, ¢ according to (28), (29), respectively.] The bounding, 
surfaces follow from (19) and the planes defining the first, 
octant (in Aj, A,, A.-space), are shown for a typical case in, 
Fig. 2 (opposite). A variety of different regions are clearly 
possible in these ternary cases, depending on our choice’ 
of the preassigned costs, subject to (7), of course. 


(c) Semple (N + 1)-ary Detection 


A particular case of considerable interest and simplicity 
(as far as the structure of the detector, e.g., (14), (15), or 
(18), is concerned) occurs when we assign the same constant 
costs Cy (> 0) to all types of “failure,”’ and zero costs to 
all types of ‘‘success’’. Then (9) becomes di” = @y, A,” = 0 
(1 # k);¥.” = — Go, and the decision process (14), (15) is 
governed simply by the inequalities: 


tdey.: TE AV) >'A,(V), | 

all(j #k); (G7 =1,---™) (, BI) 
A.V) > 1 J 

each k(= 1, --- N) in turn. For noise alone (15) is 

ply 

eee i AV < 1, Gl = ye Wo 0/2) 


p boundaries of the decision regions are obtained as 
jore from (16, 17), in terms of the equalities above. For 
nry detection, when the case ‘“‘noise-alone”’ is removed, 
first line of (31) provides the desired inequalities, and 
boundaries of the decision regions again follow from 
indicated equalities, see (19). 


Fig. 2—Regions of decision for ternary detection (NV = 3): 
S@ + N vs. S® + N vs. S@ + Nia, 6b, ¢ > 1). 


| 
|. ems or Drcrsion Errors, AVERAGE RISK 
AND INFORMATION Loss 


Vhile the section on minimization of average risk 
bs the structure of the detector and the decision process, 
shoose suitable thresholds and even to make suitable 
| assignments, it is often necessary to know the error 
ies and to assess their effects on performance 
computing the average risk. Now with each possible 
orrect decision we may define a conditional probability 


‘rror, according to 
p) oD 


= | am | WFW |S) av (33) 
| T 
= cond. prob. of calling null signal any one 
member of kth signal class, (k = 1, --- N) 
es | oe | VXFW | 8) av (34) 
r 


= cond. prob. of calling any one member of 
the kth signal class a member of the jth 
signal class (in noise). (j # k;7 = 0, --- N, 
Keay 3:i-4N)) 
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The conditional probability of correctly deciding that 
any one signal member of class k is present in noise 
(k= 1, --- N) is 


Gh y= il b(y. | VKF(V | S)), dV (35) 


- 
-1- D6. (350) 
In the situation where the case ‘noise alone” is removed, 
we have (8;""),, (7 # k); (j,k = 1, --- N). 

It is convenient now to make a change of variable and 
consider some monotonic function of the A,’s as our new 
independent variables, since it is in terms of the A’s that 
the decision regions for Yo, --- , Yv are explicitly given. 
A simple choice of particular utility is 7, = log A,, 
(k = 1, --- , N), so that (33, 34) may now be written 


ay,” aa OG. Carp (36) 


ull [zy] 
’ Xy) dx, 


GP =n] occen| me aon 
[x1 [zn] 
(Bae 7 Oar 


where Q,, Px’? are the joint probability densities for the 
random variables log A,, --- , log Ay, in the first instance 
with respect to the null hypothesis H,, and in the second, 
with respect to the alternatives H,,. Here [2,] signifies the 
decision region for x, (some typical cases are illustrated in 
Figs. 1 and 2, for the transformation x; = A,, rather than 
x, = log A,.) These probability densities may be written 
in terms of the original V for additive and independent 
signal and noise as 


, ty) dx, tes diy, 


ee Ua. 


(37) 


Ques, -+ yay) = [ WAY) TI ale; — log a) av (88) 


Baas Pe as tw) 


y (39) 
I] 6(z; — log A,) dV. 


j=l 


- | Wav - 8”), 


(A specific example in the gauss case is given in the follow- 
ing section.) 

With (33)-(36), we can now write the average risk (8) 
or (12), minimized or not, as 


N N 
Riles by eg a” ee) ae, Oe 
it Kel=1 (40) 


N 

(k)  (k) 

== Oy: Nk 9 
k=1 


which for the case of the excluded null signal simply omits 
the terms in )<°’, and sets ®) = 0, with Ax”, (k, 1 > 1) 
equal to the costs C;’’ > 0. The expressions (36, 37) for 
the error probabilities appearing in (40) have particularly 
simple limits for [x,] when we make the (@o, 0) cost as- 
sumptions of example 3(c) above. We easily find that 
now (the primes indicate terms j = / omitted in product): 


6 IRE 
Nps 
(eee) 
4 N rj 
BP = J dey me the dr)P mca b's) 
(j #k:j,k >, 
and 
. 0 
Bey ee ie da, °°: ie daz Pa (hip Oy): Ee) 


When the noise class is omitted, (41) and (48) do not 
apply, and the lower limit on the first integral of (42) 
becomes — © instead of 0. For the general cost assump- 
tions (9) we must use (16, 17), or (19), all k, to specify the 
boundaries on A, and hence on log A,. General results for 
the ternary case (Example 3(b)) follow at once from 
(24-26) or (27-29) with the aid of Figs. 1 and 2. 

Once the a”, (85"), have been found for our opti- 
mum system, we can assess its performance not only 
from the risk point of view, e.g., (40) but also determine 
the average information loss or equivocation which in- 
evitably occurs when a definite decision is made under 
the present conditions of noise and finite observation 
periods. The actual amount of information conveyed, on 
the average, is 

N 
Hy = —H(c, 8) —qlogq— 2p logy, (44) 
and the equivocation H(c, 6) may be computed from the 
definition of the a’s, 6’s, p’s, etc., to be” 


H(c, 6) = — s > Pov, S;) log [P@:, 8;)/P@:)], (45) 
where 

P:; S) = p86; #0), or ga, (j = 0), (46) 
and 


ga” + pl — 8) + Ov? @ =O), 
3 i200 
Py:) = (47) 
| > a? + ol — Da”), @ = 0). 
7#0 k#0 
Following section illustrates above discussion, with specific 
example involving an optimum (N + 1)-ary decision: 


An EXAMPLE 


Let us consider the situation in which the null signal, 
or any one of N arbitrary, non-zero signals (and noise), 
identical in structure but differing in amplitude, may be 
present at the input. Detection is assumed to be coherent,” 


§ For the expression for equivocation in the binary case, see D. 
Middleton, “Information loss attending the decision operation in 
detection,” Jour. Appl. Phys., vol. 25, p. 127; 1954. 
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the signals and noise are additive, and the costs of correei 
and incorrect decisions are taken to be zero and unity 
respectively. The noise itself is assumed gaussian, with ¢ 
known variance matrix, see (67) e¢ seqg., and we wish the 
explicit structure of the optimum detector, with the assoct 
ated minimum average risk. Here we make use of the 
specific distributions P\” and Q,, derived for this case it 
the Appendix below. 

The kth hypothesis class contains but one signal, 0) 
amplitude a,,. Let us accordingly order the amplitudes fo: 
(k = 1 +++ N): 0 < Go, < Gos 2" << Go, = on eee 
observe from (75) et seq., that the signal matrix Sy become; 
here specifically 


a 


where a, is the column vector [d@o:, -+- Gov]T, ®, (s, 8) t 
the structure factor [see 68] common to all the signal, 
in this example, and f¢ indicates the transposed matrix 
Since the matrix a» at, is singular, the joint distribution: 
Q,, P,, aS in (77), (80) take on special forms which ar 
best found if we return to their Fourier transforms (76) 
(79). Here we note that the quadratic from &f (a> a7o) 
can be reduced toa single squared term by two successivi) 
changes of variable: | 


Et(acat)E = ZTAZ = UfctAcU = Ui, 
( cele » OF NDS 
cU, and A = || 1 ||, and the (VXN, 


Sy = ©,(s, s) || Qo; Ao; || = 4,(s, 8)AATo, 


(49 


where Z; = a; &;, Z 
matrix ¢ is given by 


1 Q------------- 0 : 
Oe | 
e “ 
lie 8 ee 0 
—)] -1l - ==) ole le SST | ee 
_ 
! 0 mim 0 (50. 
= 

Oa a ~1 

(any [= 1, N) 
Eq. (77) for Q,,(y) then becomes 

Q,(Y) = aor iE [exp {—(1/2)u7®,(s, s)i= ims /o,) Wot 


N 


as Pe : du; 
: ne Ob: iL exp {tu ;(y:/aor at Yy;/Qo;)} a | 


7=1 2n 
I 5(y; 


where the prime now indicates that the /th factor in thi 
product is omitted. The result for P,” (y) is the same 
with y; replaced by y; — ao;@o, ® in (51). 

The error probabilities (81)-(83) are next found with thy 
help of (51) and the expression for P,”. First evaluatin; 
the integrals involving the delta functions, we find that 


= Aoi(2ab)*”” exp [ — y;/2a;®] yj Qo;Yi/A01), (51 


i} ee 


d(y; ay o;Yi/Ao1) dy; 


, co 


A i for adoyi/do. < y, + A, — A; (52) 
0, otherwise. 
e inequality on the right is equivalent to 
Wicket Ores SAGs 1) Ym> Ci. Lge), (53) 
here 
bi = 2401(do; + Aor) ®,(8, 8) + [Gor/(do; — ao1)] 
| Log [ui/asl, (dor # o,). (5A) 


he product II} of integrals (52) which appears in (81)-(83) 
thus unity over a range of y, extending from the lower 
mt C,. = Max; C,;, (7 < 0), to the upper limit C,, = 
lin; C,;, (7 > 0, and is zero outside this interval. 

With the arbitrary index / set equal to 7, for convenience, 
b1)-(83) become finally 


Ci+ 
|? = avieray” | exp (—y"/2a3,®) dy, (65) 
| (CZ =or—Al;) 
nd 
Ci+ 
HP) = avira)” | 

(C;—or—Aj;) 

exp [—(y — ao;Gox8)’/2a;] dy, (56) 


1 which the larger of the lower limits is to be used. 
imilarly, with 1 = k, (82) becomes 

| Cot 

ie =—oanh) ! exp [—(y — a0.)?/2a;,.8] dy, 

| (57) 
‘ith 


(7 # k). 


fox = minj [GoGo ;P/2 — do, log bu; /Qo;]; 


To show how these relations may be used in a simple 
ase, let us assume the amplitudes to be uniformly spaced, 
) that a); = 7A, where A is the (normalized) “quantiza- 
on” interval, and let us assume also equal a priori 
robabilities, so that u; = 1 (j = 1, --- , N). Eqs. (55)-(57) 
iow easily reduce to" 

- = OG + 1/2)d] — Oly — 1/2)a]}, 
a” = {1 — @UN — 1/2)d]}/2, 


Y = HOG — & + 1/2)d] — Oly — & — ad], 


(58) 
(59) 


10 These results may alternatively be derived from the basic in- 
qualities (14) defining the optimum detector. One notes that for this 
pecific case all of the likelihood ratios depend upon the single random 
ariable @(v, s), (69). It is therefore merely necessary to calculate 
1e (one-dimensional) distribution of this quantity, after which the 
1equalities (14) give directly the limits on the integrals defining the 
rrors. Usually, however, the multiple integrals by which P, Q, and 
1e error probabilities are defined, do not “fold” conveniently, and 
1e general approach of the previous section must be used. 


55 Middleton and Von Meter: Multiple Alternative Detection Signals ih 


Nace 
.N 


is etree 
eee L crior, 


‘ (j=, — (60) 


By = {1 — @[(W — k — 1/2)a}}/2, 
(k= 1,---,N — 1), (61) 


where d = A [®(s, s)/2]'” and @(a) = (2/n'”) fie” dy 
is the familiar error integral. Since the N signals (and 
noise alone) have equal a priori probabilities in this 
example, we write p, = q = (V+ 1)',k =1.---N. 
Because of the simple cost assignment, the Bayes risk 
here becomes the product of (NV + 1)~* and the sum of the 
error probabilities (58)-(62). Noting that 


N Nel N 
a oe a 
0 p=0 


ll —O@/4/2, (ie. (63) 
we find minimum average (or Bayes) risk now simply 


Ryle, 0) = aq tl - O@/2}, 


We observe from (64) that the shape of the Bayes risk 
curve (vs.d) is independent of the number of hypotheses or 
signals, the latter entering merely as a scale factor. More- 
over, as N is increased, the absolute Bayes risk becomes 
entirely independent of the number of possible signals. 
This is a consequence of the simple costs, a przorz signal 
probabilities, and signal structure assumed in this example, 
but is clearly not true in general. Thus, as N is increased 
the number of different kinds of error increases as N’, 
but here sum of error probabilities increases only as NV. 
Since average risk equals (V + 1)° times latter, it 
becomes essentially independent of N for, say, N > 10. 


1 


a) 


(N 2D.) © (64) 


1.0 
08 Spree 6 J 
aj= 1, (i=I,...N) 
(W_fl (k+j) a ate 
06 — Cio tkejyy (nk=O-N) 
AA 
aire A = Quantization interval a= 
_|Re wer{!-® 2 
02 | 
f ina 
(@) O05 10 20 CXS) 30 


1.5 
d=A\6(s,9//2 
Fig. 3—Bayes risk for the coherent multiple alternative detection of 

N signals in Gaussian noise. 


Fig. 3, above, shows variation of Bayes risk with d. As 
quantization interval becomes larger, the risk approaches 
zero as expected, since the more widely separated the signal 
amplitudes, the easier it is to discriminate among them. 
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The dependence of d on the number of samples taken (or 
on 7’, the length of the sampling period if continuous 
sampling is used) is governed by the structure factor 
P(s, s), as in (68). For example, if the samples are un- 
correlated, ®(s, s) is proportional to 7, and d ~ AT". 
Thus, for a given N, if the minimum detectable quantization 
interval A,,;, 18 defined as the A corresponding to some 
fixed fraction of the risk R (see the discussion of minimum 
detectable signal in reference 1), we note thatA ,;, 07”, 
which is characteristic of coherent detection in general’. 

Finally, if ‘noise alone’’ is excluded as a possible input 
condition, so that detection becomes the process of dis- 
tinguishing among N (> 2) nonzero signals, Equations 
(58), (59), (62) no longer apply. Equations (60), (61) still 
hold, however, except when 7 = 1, in which case we have 
the new relation 6; = {1 — @ [(k — 3/2)d]}/2. The 
sums of the error probabilities may be found as before; 
the results are 


Nia N 
Sed eh OS 13 
1 7=1 


N 


Dee 


2 


Si SOG 22) b= 21) 6) 

The Bayes risk is then easily shown to be 
Neal), 
Re(o, ) = (“== - @@/D}, WE, (66) 


which is zdentical with the corresponding result (64) for 
the same number of alternatives. In each case, Fig. 3 may 
be used with the appropriate scale factor to find absolute 
risks or minimum detectable quantization intervals. 


CONCLUSION 


We have given the general structure of the optimum 
detector when one of an arbitrary number of different 
possible signals (including noise alone) may be present. 
The evaluation of the minimum average risk is indicated, 
and illustrated with a simple example in the N-ary case. 
However, the treatment is by no means limited to coherent 
detection and hypothesis classes containing but a single 
signal. The extension of the detailed analysis of the section 
on probabilities of decision errors, average risk, and in- 
formation loss to include weak signals and incoherent 
reception may be made directly by obvious extensions of 
the technique used in the binary or single-alternative 
theory’. For general cost assignments and multiple signal 
types, analytical difficulties increase rapidly with number 
of hypothesis classes, as expected, although ternary and 
quarnary cases appear tractable by direct methods. 


APPENDIX 
Distributions of log A, for Coherent Detection in Normal 
Noise 
As an example of some interest, let us obtain the density 


functions Q,, P,’’ when a set of arbitrary and distinguish- 
able signals S“’, (k = 1, --- , N) are detected coherently” 


in additive normal random noise, with the additioné 
constraint that each signal class (k > 1) contains but on| 
member. For this situation it is possible to specif} 
Q,, P. in closed form for all signal strengths and fc 


arbitrary waveforms. We begin by writing 
(WAV — S)), = Qn? | KP 
-exp {—2[VIK™'V — 2ViK™'s,a0.¥'” 
+ auvst.K's:J}, (6 


where we have made the usual normalization:'* S“ 3 
VW”? dorSz, With do, = Ao,/(2y)’”. Here Ao, is an amplitud| 
scale factor for the kth signal; ¥ = (Vx), the mean nois) 
intensity; K is the variance matrix || (Vx): (Vw)m) |} 
(l, m = 1, -++ , n) where (Vy) = 0 here; V is a colum# 
vector, and Vj its transpose, while | K | is the determinan, 
of the (n X n) symmetrical matrix K. Normalizing \, 
according to V = y’”’v, and writing” 


viK ‘ys; = ,(v, 8;); s{;K ‘ys; = ,(s;, ,), 


we find that log A; (V), ef. (10) in (38), (89), becomes 
log A; = log wu; + do;®,(v, s;) — a5;®,(s;, s;)/2, 

(7 = Loewe NG Li; =D; /Q. 
Since 
atv; — log A) = [exp [i (e; — log A] 46/2, (70) 
we can apply this to (38), use the transformation y; = 
a; — log u; + ao; ®;;/2 = x; — A; (whose Jacobian iy 
clearly unity), and in place of Q, (a, --- wy) write | 


Quy: + ys) = [ WCW) av 


fo foo(—0 Een) 


“exp @ S ian é,vi(K Ys, d& +--+ déy (2), (71) 


where W,, (V) is found from (67) on setting s = 0. Further- 
more, | 


Ey pecayici 5 a Ew)a 


+f exp [iRty]Qn ++ ys) din + dy (72) 


oes [ en” [Kpelle naire 


-exp [—(1/2)vtK7'v + ivta(s)], 


is the characteristic function corresponding to the dis- 
tribution density Q,, and (73) follows from (71). The 
column vector 2 is % (£) = )o7j-1 a; &; (K * W)s,. The 


(73) 


1 Notation is that of second and third papers of reference 2. 


bd 


Itegral (73) is readily evaluated” to give 


Fy, +++ Evlo = e [—(1/2)at@(K/Wa®], (74) 
7 ere 
KW = DD audnkdilst:K ysl 
= E:€1A0 joj. = ETSyé, (75) 
id the symmetrical (VXN) matrix Sy = || do; do, 


1;K ys: || = || @osdor ®;2 || = || (Sx)s2 ||. The character- 
[tic function (74) becomes explicitly 


Fy(é, +++ Exo = e [—(1/2)EtSyé], (76) 
om which the various moments of y;, ¥2 , -** , yy are 
adily obtained in the usual manner by differentiation 
th respect to the é’s. The other member of the Fourier 
ansform pair, (72), is 


Kw) = fo f Ce L-aty)) 


-Fy(é, +++ Exo dé: «++ déy/(2m)” — (777) 
= (2n)"” | Sy |"? exp {—4ytSy'y}, (78) 


his last with the help of the literature.” 
The distributions P,” are obtained in the same way. 
ine finds finally for the characteristic function 


| YH. Cramér, ‘‘Mathemetical Methods of Statistics,’ Princeton 
lniversity Press, Eq. (11-12.1a), p. 118; 1946. 
| 
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Py& +++ Ey)p = exp [ZEt(Sy). — 5ETSyé] (79) 


where (Sy), is the kth column of Sy. The Fourier trans- 
form of this is easily shown to be” 


PPG a a Yn) = (Qn) *”? | Sy ae 
-exp {—alyt — (Stw).)Sy' ly — (Sw)e]}. (80) 


Observe that because of transformation from x to y, 
probabilities of error (41)-(43) become specifically here 


A oo { ujt+Ai—Al 
a; = fs avy TT iy} ae Yn); 


= Ais —AN 
(Bo) os [ OU aie / dyn TAG PU ray 
; ios) N yjt+Aj—Al 
(Br yi, = i iy TT ii iy POW, ate Yn) 
A l=1 J—o 


with (78) and (80) for Q,, P\. When “noise alone’ is 
excluded as a possible input, (81), (82) do not apply, and 
the lower limit in (83) becomes — ~ instead of —A,. 

The results of the section on probabilities of decision 
errors apply equally well for continuous sampling: matrix 
Sy is then composed of elements which are double integrals 
between limits (0, 7’), instead of double sums, cf (68). (For 
examples, see discussion of continuous sampling in second 
and third papers of reference 2.) 


A Note on the Envelope and Phase-Modulated 


Components of Narrow-Band Gaussian Noise’ 


ROBERT PRICE 


| Summary—tThe autocorrelation function and power spectrum of 
ie phase-modulated portion of narrow-band gaussian noise is 
udied and specific computations are carried out for the case of a 
ctangular noise spectrum. Related results are given for the noise 
ivelope. General asymptotic expressions for the spectra are ob- 
ined, together with experimental verification. 


HEN we are dealing with a gaussian noise n(t) 
having a narrow power spectrum N (w) about some 
representative midband angular frequency p, n(é) 

ay be represented as the product of an envelope function 
(t) and a phase-modulated function y(d): 


* This work was supported jointly by the Army, Navy, and Air 
ree under contract with the Massachusetts Institute of Technology. 
+ Lincoln Laboratory, MIT, Cambridge, Mass. 


n(t) = Ry; yO = cos [pt + a], (1) 
where Rice’s definition’ of R(t) is employed. The power 
spectra of R(t) and y(t), or, equivalently, the respective 
autocorrelation functions ¢g(r) and ¢,(7), may be found 


from the autocorrelation function ¢,(r) of n(é).” 
Uhlenbeck® has shown that 


n(7) = RORE + 7) = $,(0)[2H(c,) — (1 — 07) K(o,)] 


T 2 ‘ 
a 5 Pn(O)2(— 1/2, Ses [es OF); (2) 

18. O. Rice, “Mathematical analysis of random noise,” Bell Sys. 
Tech. Jour., Vol. 23, p. 282; July, 1944. And Bell Sys. Tech. Jour., 
Vol. 24, p. 46; January, 1945. Section 3.7. 

2 See Appendix for derivations of ¢,(7) and ¢,(7). 

3G. BE. Uhlenbeck, “Theory of the random process,” MIT Radia- 
tion Lab. Report 454, October 15, 1943. 


10 IRE TRANSACTIONS—INFORMATION THEORY 


where AK and / are complete elliptic normal integrals of 
the first and second kind, respectively, and ,/’, is a hyper- 
geometric function. c, and ¢,(7) are given by 


,tA(7) = 


. — 


i(w—p)T 


o,realand > 0. (8) 


dw; 


v0 


I 


(0) = n(n $a) = [ NUwde'*” ds 


= ¢,(0)o, cos [pr + A(z)]. (4) 


Thus o, may be considered to be the normalized envelope 
of ¢,(7). 

Using MacDonald’s result* for the second joint prob- 
ability density of @(¢) and 6(¢ + 7) we have found 


o(7) = y(dy(t + 7) 
Z Ee al eo KG) 


oF 


| cos [pt + X(7)] (5a) 


= ae oF',(1/2, 1/2; 2; 6?) cos [pr + X(7)]. (5b) 
The series expansion of (5) is 
¢,(7) = —s cos [pr + X(z)] 
[ heel Se 
es iG + 1)! eee 


Series (6) is identical, to within a proportionality constant, 
to a result found much earlier by Van Vleck,’ and later 
by Middleton® and Davenport,’ for the autocorrelation 
function of clipped and filtered narrow-band gaussian 
noise. Middleton® has also found the closed form (5b) 
for the slightly restricted case of N(w) symmetric about 
p, so that substantially the only new result in the -pre- 
ceding is the expression of ¢,(7) in terms of elliptic 
integrals. * 

It is worth noting that (2) and (5) also apply to cross- 
correlation functions between the envelopes or phase- 
modulated components of two noises n,(t) and n,(t) 
having a gaussian joint probability distribution, provided 
that the cross spectrum ®,.(w) is narrow-banded about p. 


Dials) = 5 = 


We need only substitute, in (8) and (4), 
¢,(0) and ©;,(w) for N(w). 

MacDonald* has remarked that the second joint 
probability density for the phase @(t) appears to provide 


n(t)n(t + 7) Cm a) aa (7) 


Vind) for 


*D. kK. C. MacDonald, “Some Statistical Properties of Random 
Noise,’’ Proc. Camb. Phil. Soc., Vol. 45, p. 368; 1949. 

> J. H. Van Vleck, ‘“The spectrum of clipped noise,” Harvard RRL 
Report 51, July 21, 1943. 

HIDE Middleton, “Some general results in the theory of noise 
ou nonlinear devices, ” Quart. Appl. Math., Vol. 5, p. 445; 1948. 

B. Davenport, Jr. “Signal-to-noise ratio in bandpass lim- 

ie Jour. Appl. Phys., Vol. 24, p. 720; 1953. 

8 W. Magaous and F, Oberhettinger, “Formulas and Theorems for 
the Special Functions of Mathematical Physics,’”’ Chelsea Pub. Co., 
1949. 
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more information about the spectrum N(w) of n(é) than 
does the second joint probability density for the envelope; 
R(t). This observation applies to the corresponding auto- 
correlation functions ¢,(7) and ¢z(7), as well. Comparing 
(2) and (5), it is seen that ¢z(7) is a function of only the 
envelope of ¢,(r), while ¢,(7) depends on both the envelope 
and phase \(r). In fact, if N(w) is symmetric about p, 
\(r) is 0 or 7, and ¢,(7) can then be recovered completely 
from ¢,(7). 

We have made a detailed investigation of the spectra, 
of y(é) and R(t) for the particular case where N(w) is. 
confined to a narrow rectangular band. From this point) 
on, all frequencies will be considered cyclic rather than: 
angular, and all spectra double-ended. Let n(¢) have a 


uniform spectral density N,/2 in the regions (fo — 3, 
fo + 3) and (— fo — 3, — fo + 3) and be zero elsewhere: 
Then | 

on(7T) = No a cos 2rfo7, 


and we find’ for the spectrum S,(f) of y(0): 


SWS) = 76 u(f’) | 


a 


[aye - 75 
al(a--+ 120)! 


t=M(f') 
! 
mL) pe Or eet Ne fe 2s eae 
aS! en? the 2a 
7=0 fi 4 
where sf? = 2s his | etinale ee 5 is a binomial co= 
efficient, and 
) Ue oct | 
u(J a ee 
0; elsewhere 
Leet 
M(s’) = ae | 
largest integer < ° tae elsewhere ( 
m(i, f’) = largest integer < a (9) 


Van Vleck® has found an approximation to S,(f) by using. 
the first few terms of (8), but the convergence is poor. 
Likewise, he and Middleton’’ have found approximations 
to S,(f) for gaussian and optical n(¢) spectra, which may) 
converge better. In the rectangular case, it seems easier 
to achieve good accuracy, at least for values of f’ near 
the original spectral region, by performing Fourier trans- 
forms of ¢,(7) numerically, using the closed form (5a), 


* We have made use of D. Bierens de Haan, “Nouvelles Tables 
d’Integrales Definies,”’ G. I. Stechert and Co., New York, 1939. 
Eq. 15, Table 159. 

10 ED), Middleton, “The response of biased, saturated linear and 
quadratic rectifiers to random noise,” Jour. Appl. Phys., vol. 17, 
p. 778; 1946. 
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ther than attempting to sum the series (8). Values of 
,(f) which have been obtained by numerical integration 
*e presented in Table I, and a graph of S,(/) is shown in 
ig. 1, both above. It has been found that 89.89 per cent 
the power in y(t) is contained in the original spectral 
gion of n(t). 
The similarity between (2) and (5) enables the spectrum 
z(f) of the envelope R(t) to be found with little additional 
bor, once the computations necessary to obtain S,(/) 
ave been performed. For the rectangular case considered 
pove, we have obtained the values of S,(f) presented in 
‘able II (opposite). Fig. 2 (opposite) is a graph of Sz(/f), 
yr which an approximation has appeared elsewhere.”’ A 
Res similar to (8) can be developed for Sp(/): 


[ 
J 
» | (wana -- 754) | 
+ 2, Gi = 
m'(i,f) ; oy ee tere 
2 (ir) +3 peo (10) 


uJ. L. Lawson and G. E. Uhlenbeck, “Threshold Signals,” vol. 24, 
‘IT Radiation Lab. Series, McGraw-Hill Publishing Co.; 1950. 


TABLE I TABLE II 
ee Male Nessa Sy) a Sr(f)/No 
0.0 0.228 0.0 (x/2) 8(f) + 0.415 
0.25 0.226 +0.25 0.315 
0.50 ae +0.50 0.213 
0.0228 +0.75 0.110 
0.75 0.0153 +1.00 0.00675 
1.00 0.0093 +1.25 0.0035 
1.25 0.00495 +1.50 0.0016 
1.5 0.00278 +2.00 0.0003334 
2.0 0.00124 +2.50 0.0001 
2.5 0.00064 
20 18 10 0.5 0 0.5 10 15 AG) 2S se ae 
2 ee a 7 | 
| | 
| | 
| | 
et Coan 4o.2 ole 
t 
Sy it pool 
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ji a3 
bit 0.1 
ot 
sf 4005 
} oak 
eee os, (sea Ey R 
ions BO ar TG =o15 5 OG ia Le EG 25) -25 -2.0 -1.5 =1,0 -0.5 0 0.5 1.0 15 2.0 2.5 
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where 


M'(f) = smallest integer > | f | 
(11) 


m’'(i, f) = largest integer <7 — | f | 


and 6(f) is the Dirac 6-function. 

When N(w) is symmetric and has a finite normalized 
second moment A about p, asymptotic approximations 
to S,(f) and S,(f) can be found. The procedure is to 
expand (2) and (5) in power series in ¢, and Fourier trans- 
form term-by-term, making use of the convergence of 
high powers of o, to gaussian functions. Using Stirling’s 
factorial approximation in the coefficients of the resulting 
series, and approximating the summation by integration, 
we obtain: 


: A 
lima SCA) = my (12) 
3A 
lim S27) = 3 1 FP (13) 
where 
[ @- DN de 
5 es ae (14) 
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Comparison of (12) and (13) with the values given in 
Tables I and II shows that, for rectangular N(w), these 
approximations are quite good, even when /’ and f are 
not very large. 

J. Craig and E. Manna, of this Laboratory, have 
conducted an experimental investigation of S,(f) for 
N(w) having an approximately rectangular spectrum. 
Their results, together with the theoretical curve for an 
ideally rectangular N(w) having the same normalized 
second moment, are presented in Fig. 3. The agreement 
appears to be excellent in the regions where (J 2) is accurate. 


APPENDIX 


Derivations of dp (7) and ¢,(7). 

The starting point for finding ¢z(r7) or ¢,(7) for narrow- 
band gaussian noise is given by the joint probability 
density distribution for the envelopes R,, R, and phases 
6;, 02 of n(t) = R(t) cos [pt + O(f)] at times ¢, and ¢, = ¢, + 
r, respectively:* 


SRE PaS (5 2 2 
p(k, Ro, 6, 9.) = de A exp DA {Zolt, + Zokts 
— 22,R,R, cos [6. — 0, — xl} (15) 
where 
f= 25 ee 2.) = oOo (16) 
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and o, and X(r) are given by (3). | 
¢r(t) may then be found by performing the aa 

integration 

eee { 
p(T) = Intalites 4 


} i 3 / : ( = 1 ») dO, dR, d 2 
0 0 = x ] ) ) d é dé R 


| 


ee’ , : | 
Rice’ has performed the integrations over 6, and 6,,' 
obtaining | 


I 
; 

| 
| 
¢ 


p(R,, Re) = / (Ry, Re, 61; 6:):d0, dO, 


2 2 
Ef) wo (- 


aisles 
A A 


(18) | 
where J, is the modified Bessel function of the first kind, 
of zero order. Using elliptic integrals, Uhlenbeck* has’ 
carried out the remaining two integrations to obtain (2). 


$,(7) may similarly be found, after first noting that 
gi(7) = y(Oy(t + 7) 


_ cos [pr + X(2)] 
2 


cos [8 — & +7] (19) 


by using trigonometric expansions in (1) and averaging. 


_ cos [pr + X(0)] 
2 


elle ae 


-p(Ri, lvoe 61, 63) dé, dé, dR, dR, (20) 


facDonald* has performed the integrations over R, and 
pb, obtaining 


— 06, + X(7)] 


e 


ip(4,, O2) = / / pCR, Rz, 61, 02) dR, dR, 
1) 0 


Summary—It is shown that for any noisy channel without 
mory having only a finite number of received signals, the error in 
smitting information at a rate H < C and using uniformly good 
ides of length n is bounded by an expression Fe ?”‘°~”)” where 
.and B are constants depending upon the channel parameters but 
pt upon A orn. 


INTRODUCTION 


T HAS BEEN known since 1948 that, given a channel 

having capacity C, it is possible to transmit infor- 
~~ mation through the channel at any rate jess than C 
ad to receive this information with an arbitrarily small 
robability of error. There are two general methods of 
pnstructing codes which will achieve this. One is the 
1ethod used by Shannon in his first papers on information 
neory. The second was given by the present author in 
is doctoral thesis.” Both methods show that by using 
codes” in which the ‘code words” are sequences of 
icreasing length n, the probability of reception error 
sr any fixed rate of transmission H < C goes to zero as 
goes to infinity. Except for a very few specific cases, 
owever, no general estimate of the rate of approach of 
ne error to zero has appeared. 


| DERIVATION 


Let {X, Y, p(y | x)} be a channel without memory, 
ith capacity C. Then it is known’ that for any H < C 


_* The results of this paper were obtained while the author was 
rking at the Bell Telephone Laboratories, Murray Hill, N. J., for 
e summer of 1954. At the same time, C. EH. Shannon obtained by a 

ethod distinct from ours substantially the same results, plus pre- 

ninary results concerning a lower bound for the error. Tt was in- 
nded at that time to publish one paper containing all these results. 
nee then, however, Dr. Shannon has informed the author of several 
1portant new results which he has obtained. Under these circum- 
ances it seemed more appropriate to poe these results separately. 

+ MIT Lincoln Laboratories, P. O. Box 73, Lexington 73, Mass. 

a C. E. Shannon, “A mathematical theory of communication,” 

ell Sys. Tech. Jour., vol. 27, pp. 379-423, 623-656; 1948. 

2 A. Feinstein, “A New Basic Theorem of Information Theor Vv; 
ept. of Physics, MIT, September, 1954; or equivalently, IRE 

PONSn Gelade, “Title,” September, 1954. 
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le pres e + sin”? 8) 
— An’26 <0 (1 = 6G : 


(21) 
where 


Tv eae 
ee < ‘si. #6. < : and 6 = (z,/2) cos [@, — 6. + A(7)]. 
The results (5a) and (5b) are obtained by performing the 
remaining two integrations.” 


® Details may be found in R. Price, ‘Some Results in the Study 
of Narrow Band Gaussian Noise,” MIT Lincoln Lab. Technical 
Report No. 75, Jan. 26, 1955. 
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AMIEL FEINSTEIN} 


and e > O we can find n(e, H), such that for each n > 
n(e, H) we can find more than 2” sequences wu; = {ai,---, 
x} and corresponding sets A; of sequences v of length n 
of y’s such that the A, are disjoint and p(A; 
The interpretation of this statement is that we can transmit 
information through any channel with arbitrarily small 
probability of error if (1) we transmit at a rate H < C, 
and (2) if we code our information into sequences of 
sufficient length n(e, H). 

We wish to obtain an upper bound for e as a function 
of n, H and the channel parameters. We start with the 
inequality” 


1 ae FEN) 
€ << B (VA ar ahs “5 
where 


BoSa = b> and) Awe oe 


Furthermore, 6; and 63 are functions of n and e and ¢:, 


being defined as follows: 


= Prob. {pu |v) = 


(ala) 
< Oe aa 


pay | Yeap 


and 
= Prob. {p(u) > 277? 


where all probabilities are taken as those for which 
FEO.) TO Oe 

Let Z,, ---, Z, be a set of independent random variables, 
identical to — log p(x | y). Then we may put 


== Prob: 2 a re + 4, TOGO ah 


and E(Z,) = = H(Z,) = H(X | Y). We can likewise 
express 62 in similar fashion. To estimate 6, and 63 we 
use a theorem of Feller® which, for our purpose, may be 
stated as follows. 


“Generalization of a Probability Limit Theorem of 


3W. 
Math. Soc., vol. 54, pp. 361-872; 1943. 


Cramer,” 


Feller, 
Trans. Amer. 
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Theorem. Let {X;},7 = 1, 
identically distributed, 
S = > 1.1 X; and let 

F(x) = Prob {S — nE(X,) < z}. 
Put o” = E({(X, — E(X,)}) and take 


y > UP exe — A) f 
on 


-,n be aset of independent, 
bounded random variables. Let 


Then if 0 < Ax < 1/12, we have 
1 — F(aon'’”) = exp [—1/22°Q(2)][{1 — B(2)} 
+ 0d exp (—1/2z2’)], 


where 


1 12a 
1Q@ | Sz (; = | 


®@) = gam [ew [-v/2) ay 


and 


1/2 


€ 
In our case, 7 = +n” and we may take* 
o 


2 sup | X, — H(X,) | 
7 ; 


a 172 
on 
Ti 1 —27/2 
Now asa, 1 — (x) NOES e ; hence for x > a 
Clie 


for suitable a, 1 — ®(x) < 7 - Now 


2 sup | Xi — £CX%) | er - 


i} 
o 


if we take e, so small that Av < we have: 


I 
12’ 
Fe ne; Gra 1 1 

re elas, ee 
gi OXe { ee E — 12n2) th} MES) 


oe per HCO EL zh 
oO €) 


provided that Sn? > a. Let us take ¢ = Aji(C — HH). 
oO 
If A, is sufficiently small, 


6Ax | il 
ee 
7(1 — 12d) Res 4’ 


say, for any H < C. We can now say that, given a value 


4For \ <o it is clearly sufficient that the channel have only a 
finite number of received signals, since the number of transmitted 
signals we always assume finite. See also the reference of footnote 2 
for various remarks on this. 


September 
2 | 
of H, we have 6, < ¢ ™"°°"”", where B, = ig pro- 
vided that n is so large that 
AC = H) mi > a 
oO 
and 
—1/2 18 sup | Xy ae E(X;) o \ < 1 
n : rea) 


It is, however, easy to remove these conditions, as! 
follows. Both conditions require that the smallest value 
of n, for which both conditions are satisfied, is pro-, 

1 1 
(Cay? J 
when it becomes a true bound for 4; is some) 
constant K, 0 < K, < 1, independent of C — H. The 
expression 


portional to - This implies that the value of! 


—Bin(C—H)? 
é 


ak ers 

Ky 
5 : =Bin(C=H)2 =m 
is then certainly a bound for 6; whenever e **"' 18.) 


Further, wherever the latter fails, the former is greater) 
than 1, and certainly greater than 6;. Hence, without; 


1 Cnn ee 


qualification, 6; < K, 


1 —Bon(C—H)? 
=k? 
H) for suitably small A,. 


We can similarly obtain 6; 
é, = A,(C — 
Nowe < z (VA + V/38;)". Since 


, taking | 


—n(C—H—e€1—€2) 
A=2 we 


ules aa= As 2(C—H)?2 


An ya A,—A2)(C—H) x Deere Oa, Oa 


i} 


as long as we take A, + A, < 1. We therefore have} 


{aa fee . ite 
e 2"~")" for suitable positive D. However, 


Bw 
1 
ar 


to keep 3 bounded we want 6, bounded away{ 


—Ben(C—H)? 


from 1. Now 63 . If, say, we Wane < 28 


1 
SK B 


r 6, <=, we must keep n(C — H)’ constant. But, as be- 


fore, we can eliminate this restriction by multiplying 
de Pn-* by a suitable factor. We finally have 
e< Fe?" > where F and D depend upon the chan- 
nel parameters but not upon n or H. | 


° There should be no confusion between e, the probability of error, 
and e, the base of the natural logarithms. 
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HIS BIBLIOGRAPHY is an attempt to collect 
| a reasonably complete set of references pertaining 
primarily to fluctuation type noise, characteristic 
if shot and thermal noise, up to the year 1954. Of necessity 
hese references touch upon such diverse fields as electrical 
atl aerodynamics, quantum mechanics, math- 


matical statistics, and pure mathematics. Although 
he material has been divided into a number of broad 
vategories, it should be pointed out that many references 
could appear with almost equal validity under more than 
re heading. The classification is somewhat arbitrary and 
he reader is advised to consider related topics in all 
tases. Within each category the items are arranged 
thronologically with respect to their appearance in print. 
(he attempt has been made to survey as accurately as 
possible the field in English, and to include as many 
yertinent and accessible foreign-language articles as 
practicable. 
_ Because noise is the relentless and undesirable associate 
»f intelligence transmitted or received by acoustical and 
1 es systems, it is essential for any proper theory of 
sommunication to provide suitable knowledge and 
methods for studying the physical properties of noise and 
its interaction with a desired signal. Not only is a success- 
ul technique of measurement required to control or lessen 
he noise, but also adequate information and theory is 
equisite to guide experiments and to interpret data. 
he purpose, therefore, of this bibliography is to present 
» coherent source of information references for the case 
of noise belonging primarily to the fluctuation type 
random normal processes), rather than impulsive noise 
such as atmospheric and solar static, the cataloguing of 
which would have been prohibitive due to the enormous 
number of publications of observations. 


| Classification Chart 


1. Source Works: basic texts, monographs, and 
auxiliary bibliographies. 
2. Internal Noise Sources: 
element noise, etc. 
. External Noise: signals received in the presence of 
noise, control systems with statistical inputs. 
4. Noise Generation and Measurements. 
5. Impulsive Type Noise: atmospheric and _ solar 
noise. 
6. Modulation and Noise: noise and PM, PCM, 
PTM, AM, and FM systems. 
7. Radar Applications: jitter, clutter, glint, scin- 
tillation, ete. 


tube noise, circuit 


+ Westinghouse Electric Corp., Air Arm Division, Friendship 
nternational Airport, Baltimore 27, Md. Univ. of Maryland, College 
ark, Md. 
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A Bibhography on Noise 


P. L. CHESSINt 


8. Noise, Communication and Filtering: data 
smoothing, Wiener theory, etc. 

9. Statistical Theory: general studies of random 
noise, generalized harmonic analysis, etc. 

10. Author Index: alphabetical listing. 

11. Appendix: publisher listing. 


*2-1 


In past years, since Schottky first noted and 
correctly interpreted shot effect, the subject of fluctuation 
noise was quite mysterious even to the best of engineers 
and scientists. In 1928, however, Johnson’ * and 
Nyquist” * cleared up the subject of thermal agitation 
noise in circuits. From about 1935, fluctuation noise in 
the plate circuit of amplifier tubes has been thoroughly 
worked out” '’, and, with some further work on mixer 
noise and converter noise® *'*~’, it is possible to use 
accurate qualitative data on tube noise for receiver 
calculations. At high frequencies, however, input noise 
must be considered; this too has been evaluated” *°?~””. 
The signal-to-noise ratio of a radio receiver at ultra-high 
frequencies is primarily dependent on noise relations for 
tubes and circuits’ *’, but also one must consider the 
input signal and its transmission to and through the 
receiver. Early published works* *'*’’ on signal-to-noise 
ratio did not include induced input noise and hence was 
not strictly applicable at uhf. Until North’s paper® * on 
the quantity called “noise factor’, there was no widely 
accepted basis on which to compare qualitatively ex- 
perimental or analytical results. The extension of the 
analysis to include induced noise was made® ‘, and the 
interpretation of results in terms of noise factor has 
become common. As a result it was possible to discuss 
signal-to-noise ratio at uhf with considerable clarity’ °°” **. 

Since the paper by Armstrong® * drew attention to the 
possibilities of frequency modulating regarding the 
reduction of noise, a considerable quantity of work in 
this direction has been published. The earlier theoretical 
treatment of signal and noise, however, had been confined 
in the main to the case where the noise energy is small as 
compared to the signal energy. Now rigorous treatments 
valid for all signal-to-noise energy ratios may be found. 
These treatments are usually developed by methods which 
Rice’ and Frianz* *’ *~*° applied to similar problems 
and which are based on the Fourier Spectrum of the 
noise.’ 1—5, 1—8,) 1-125 9-2, 9-9, este 

In the past fifteen years a great deal of study has been 
aimed at the mathematical analysis of random noise in 
communication systems’ “°. Theoretical contributions 
have been made by many people, among whom are Rice, 

* Editorial note—All superscripts refer to bibliographical data 
which follows, in this manner: In superscript >, e.g., “?” designates 
section 2 (Internal Noise Sources), and ‘“”’ refers to the first author 
listed, etc. 
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19, 3—25, 3-32, 3—46, 


mentioned earlier,” “’ and Middleton®~ 
soos Tere 9°" The former’s papers give a unified view of 
the fundamental methods of noise analysis for both linear 
and nonlinear circuits. The latter has solved a good 
number of nonlinear problems of practical importance. 

Since the mathematics is quite complicated’ °” °~°*, a 
need for developing experimental techniques to determine 
the important statistical characteristics of random noise 
in circuits, and, incidentally, to check existing theoretical 
results and to solve problems not susceptible to theoretical 
analysis was felt in many quarters. The method usually 
consists of finding the autocorrelation function” *’ and 
by a Fourier cosine transformation determining the power 
density spectrum’ ™: °~"*. At MIT’s Research Laboratory 
of Electronics, for example, there are available an elec- 
tronic digital correlator’ ** and other devices, such as the 
electronic differential analyzer’? ** and the delay line 
filter® *’. The last two machines accomplish the trans- 
formation of the correlation experimentally obtained by 
the first. Some work by Knudtzon**° was done in this 
direction where, however, experimental studies were 
limited to the case of linear circuits. On the other hand, 
Weinberg and Kraft” *° have extended the scope of 
investigation via the correlation technique to include 
nonlinear devices. 

The determination of an optimum design for an auto- 
matically tracking radar-controlled system is hampered 
by the difficult problem of analyzing the character of the 
radar noise. The engineer who wishes to minimize the 
effect of the noise in the tracking loop, for example, may 
use the Wiener “Optimum Synthesis’? technique’ ° to 
determine the characteristics of an optimum linear control 
system. The equations which are based on the criterion 
of minimizing the rms error on a statistical basis are 
discussed in’ °. The synthesis of such a system requires 
the aforementioned autocorrelation function and power- 
density spectra of the input message and noise character- 
istics, in order that the power spectra may be placed into 
the design equations to obtain the optimum system. 
Phillips and Weiss® *° concerned themselves with the best 
smoothing of positional data for gunnery prediction for 
noise spectra of arbitrary forms, while Floyd, Zadeh, and 
others S~32, 839 extended Wiener’s theory’ ’ to 
nonstationary time series, and obtained a more general 
solution to this problem. 
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Proceedings or Transactions? 


Questions frequently arise—in [Editorial 
Board discussions and among the reviewers 
of PROCEEDINGS papers—as to what should 
be published in the PRocrEEDINGS OF THE 
IRE and what should be published in the 
various TRANSACTIONS. This statement will 
not settle the matter, but it does clarify the 
present situation and speculate about future 
possibilities. 

One aspect of the current state of affairs 
is that of the twenty-three Professional 
Groups of the IRE, twenty-one publish 
Transactions, while only ten of the twenty- 
one TRANSACTIONS appear regularly. Thus, 
in some fields of interest the PROCEEDINGS is 
the IRIE’s only facility for publication, 
and in some it is the only regularly pub- 
lished facility. As long as this situation per- 
sists, there must be a certain seeming incon- 
sistency in the publication policies of the 
Procreepvinas. In some fields the PRocrErp- 
INGS will endeavor to publish all papers 
worthy of publication, and in other fields it 
will publish only a selected few of these 
papers. It is to be hoped, of course, that 
neither TRANSACTIONS nor PROCEEDINGS 
will publish papers which are not worthy of 
publication. 

We hope that this inequity of coverage 
by the PRocEEDINGS is a temporary matter. 
While it continues, the ProcerpiNnes will 
have to cover some fields in more detail than 
is perhaps desirable and it may publish too 
little in the fields of the most active and 
effective Professional Groups. The remedy 
for this situation is obviously a continued 
expansion and improvement in the TRAN- 
SAcTIONS, and effective laison between 
editors of TRANSACTIONS and PROCEEDINGS. 

Let us now consider the specific case in 
which a field is covered both by an active 
Professional Group with a fine TRANSAC- 
TIONS which appears regularly, and by the 
PRocEEDINGS. Among all papers worthy of 
publication, which should go to the PrRo- 
CEEDINGS and which should go to the 
TRANSACTIONS? Above all, whatever the de- 
cision should be, papers should get into the 
right journal as promptly as possible. While 
an initial mistake might in some measure be 
remedied by republishing in the PRocrEp- 
INGS a paper which appeared first in the 
TRANSACTIONS, this is bad, and papers will 
be reprinted only under the most unusual 
circumstances. A technical journal is most 
valuable and most interesting to its readers 
when its contents are fresh and new. What 
about simultaneous publication in the PRo- 
CEEDINGS and in the TRANSACTIONS? This is 
pointless, for the PRocEEDINGS reaches all 
members of the IRE. The problem, then, is 
to get the papers into the right journal the 
first time, and promptly. 

It is not easy to lay down rules which will 
cover all papers. There is certainly a place in 
the Procerepines for review papers which 
summarize recent progress in a field, and for 
tutorial papers which teach new concepts 


and techniques of analysis. The PRocrrp- 
INGS asks experts to write such papers. 

Most of the material in the PRocrEp- 
INGS is submitted unsolicited. In trying to 
lay down rules for reviewing papers, the 
Editorial Board has said that to appear in 
the ProcrEDINGS papers should be impor- 
tant enough to be of general interest (to a 
reasonable fraction of the Institute’s 40,000 
members). This has been further interpreted 
by saying that papers should represent 
either a contribution of permanent value, or 
sound work of great current interest. 

In being more specific than this, one can 
only describe and comment on particular 
types of papers. 

A paper submitted to the TRANSACTIONS 
or the ProcrrnpINGs may constitute a sort 
of review or tutorial paper if it presents a 
rational approach to a new field quite clearly 
or completely, so that one unfamiliar or only 
slightly familiar with the field can use it. 
Such a paper may or may not contain im- 
portant new information. If it is really good, 
it will be suitable for the PRocrrpINGs. 

A paper may describe an important new 
invention or device. If this device has con- 
siderable present or potential importance, 
and if it has real novelty, the membership of 
the IRIX should know about it promptly 
in a paper of length adequate to explain it 
clearly. Perhaps the author will want to 
treat at length many important details 
which are of interest largely to specialists; 
he should do so in the Transactions. The 
PROCEEDINGS paper should, however, be a 
real technical paper and not a news item; it 
should be technically sound and complete 
enough so that a more detailed discussion 
can be based on it and can refer to it. 

A new experimental result, or new theory, 
should be treated much as is a new invention 
or device. 

Many papers are concerned with calcula- 
tions of great importance, about, for example, 
vacuum tubes, networks, antennas, or solid- 
state devices. If a calculation exhibits a new 
and important principle which casts new 
light on important problems, it is suitable for 
the Procrrpines. If it merely solves a par- 
ticular difficult problem by ingenious but 
special methods, it will usually be of interest 
only to specialists. In such a case, the result, 
if it is important, can be conveyed to the 
general IRE membership through the ab- 
stracts of TRANSACTIONS papers which ap- 
pear in the ProcrEpines, or through a 
letter to appear as Correspondence. 

In all cases, we should ask, ‘Should 
people outside a particular Professional 
Group be told about the contents of the 
paper? Why should they be told? How much 
should they be told to make the telling a 
real technical communication and not a 
news note?” 

Where should an author send his paper? 
If he believes that he has something to say to 
the entire IRE membership, he should send 
it directly to the Procrrprnes. If he is ad- 
dressing only experts in his field, he should 
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send it to the Transactions. To assure the 
promptest publication the author should 
make the right choice. 

Sometimes the author makes the wrong 
choice. It is up to the reviewers to be alert 
about this, and to recommend to the PrRo- 
CEEDINGS or TRANSACTIONS a paper which 
they feel has been misdirected. This makes 
some sort of liaison desirable. Several Pro- 
fessional Groups have named one or more 
of their members to act as one of the three 
PROCEEDINGS reviewers on all papers in the 
field of their Group. This makes available at 
the time of the review for the PRocrEpINGS 
the opinion of a representative of the Pro- 
fessional Group. 

The relationship between the TRANSAc- 
TIONS and the ProcreEpINGs must evolve. 
During this evolution, it is important that at 
each stage both the TRANSACTIONS and PRo- 
CEEDINGS publish promptly interesting and 
worthwhile material. The material must be 
worthwhile in each case, and perhaps the 
chief distinction must be between what is in- 
teresting to a large and heterogeneous group 
and with what is interesting to a specialized 
segment. 

J.R. Pimrcn 
Editor, IRE 


A Note on the Bounds on Auto- 
correlation Functions 


As is well known, the autocorrelation 
function ¥(7) attains its maximum value at 
7 equal to zero. Furthermore, if the auto- 
correlation function vanishes for | 7| > T, 
WMI HOP || ap) << I’ 


| Wr) | < ¥0) cos ——"—.,_ (1) 
[El +: 


where [7'/r] is the least integer not less than 
T/| 7 | 

Recently, bounds were shown to exist on 
the transient response of networks whose 
system functions were appropriately re- 
stricted.?»3 Since the impulse response and 
real part of the system function are Fourier 
cosine transforms of each other and the 
autocorrelation function and power spec- 
trum for stationary processes are similarly 
related by (2) and (3), many of 
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1R. P. Boas, Jr., and M. Kac, ‘Inequalities for 
Fourier transforms of positive functions,’ Duke 
A emer Journal, vol. 12, pp. 189-206; March, 
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2 A. H. Zemanian, “Bounds existing on the time 
and frequency response of various types of net- 
works,” Proc. IRE, vol. 42, pp. 835-839; May, 1945. 

3A, H. Zemanian, “Further bounds existing on 
the transient responses of various types of net- 
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a= [ w(f) cos 2rfr df (3) 


»se results may be applied directly to the 
tocorrelation. This analogy depends on 
2 fact that the power spectrum is a non- 
vative function. The results will be stated 
thout proof, since these proofs appear 
ewhere. It is merely a matter of substi- 
ting the appropriate variables for those 
pearing previously in the papers on tran- 
mt response. That is, one-fourth of the 
‘wer spectrum [w(f)]/4 is analogous to the 
nl part of the system function R(w) and 
e autocorrelation function W(r) is analo- 
jus to the impulse response W(¢). Thus + 
places t, ¥(0) replaces 1/C, and [w(0)]/4 
places r in the previous papers. ¥(0) is the 
tal power of the signal. 

he first result is obtained from Theorem 
of reference 2, and is similar to the bound 
‘tained by Boas given by (1) above. In 
jis case, the condition that the autocorrela- 
m function vanishes for all | 7| > 7 is 
dlaced by the less restrictive condition 
at the autocorrelation function be bound- 
by some fraction of its maximum value. 
ssulting bounds for | 7! < 7’ are, however, 
't so strong as those obtained by Boas. 
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"The second result is obtained from Theo- 
m II of reference 3. It is applicable if the 
/wer spectrum is monotonic decreasing for 
breasing positive frequencies. 
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Correspondence 


Some Thoughts on Technical 
Meetings 


As a result of my recent experiences as 
Chairman of the Organizing Committee of 
the 1954 Symposium on Information The- 
ory, | have given some thought to the gen- 
eral problem of technical meetings with 
particular regard to the relative functions of 
Professional Group symposia and the tech- 
nical sessions at the National Convention. 
IT am jotting down my thoughts on this 
matter, as I understand that the organiza- 
tion of technical meetings is a subject of 
current interest to many IRE members. 

I shall begin with a brief report on the 
1954 Symposium on Information Theory. 
The organization of the Symposium de- 
parted from current practice in two major 
respects. In the first place, all papers pre- 
sented at the Symposium were published in 
a special issue of the TRANSACTIONS OF THE 
PROFESSIONAL GROUP ON INFORMATION 
TuHeory and distributed to all participants 
two weeks before the date of the Sympos- 
ium. In the second place, ample time was 
allotted to discussion; it averaged about one- 
half hour per paper, while the formal pre- 
sentation of the papers averaged about 20 
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can be shown that the upper bound is 
est possible for all + and that the lower 
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minutes. Furthermore, a Discussion Leader 
was formally appointed ahead of time for 
each of the Symposium’s 6 sessions; function 
of the Discussion Leader was to start discus- 
sion off on the right track by making care- 
fully prepared comments about each paper. 


* Received, October 13, 1954; revision received, 
July 9, 1955. 
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The total number of registrants was 300. 
Although I did not make detailed checks, I 
would say the attendance at all sessions 
averaged between 80-90 per cent of regis- 
tration. The discussion was quite lively most 
of the time and had to be stopped by the 
Chairman to keep the meeting on schedule. 
I noticed with particular pleasure that most 
of the Discussion Leaders, Chairmen, and 
Authors attended all 6 sessions and seemed 
to enjoy the meeting. All in all, the consen- 
sus seemed to be that the unconventional 
features of the Symposium represented a 
step in the right direction. It was obvious, 
however, that a technical meeting such as 
that could not be attended by any appreci- 
ably larger audience. 

The 1954 Symposium on Information 
Theory met fairly well, in my opinion, the 
needs of the relatively small group of IRE 
members who are doing active research in 
the field of information theory. I felt that 
the discussion was fruitful in stressing the 
present state of development and in sug- 
gesting avenues of approach to unsolved 
problems. On the other hand, I doubt that a 
person interested primarily in specific prac- 
tical applications or one interested in keep- 
ing up-to-date in general terms with the 
over-all progress in information theory, 
could have learned much by attending the 
Symposium. The realization of this fact 
brought to my mind the problem of how to 
meet the different technical needs repre- 
sented in the IRE membership. 

Broadly speaking, I would say that the 
IRE members interested in each specialized 
field represented by a particular Professional 
Group may be divided into three main 
groups: 


A. Members who are doing active ad- 
vanced research in the field and who, 
therefore, badly need opportunities for 
thorough discussion among themselves; 

B. Members who are interested in specific 
practical developments and who, there- 
fore, need to keep abreast of results 
obtained by the people in group A, as 
well as by people doing similar develop- 
mental work; and 

C. Members who, for various reasons, wish 
to keep themselves informed of the broad 
lines of progress in the field, but are not 
interested in the detailed developments. 


Two points should be noted in this regard. 
First of all, the size of group A is relatively 
small in each specialized field, while groups 
B and C are rather large. In the second 
place, a person who fits in group A for one 
particular field will in general fit into group 
B or C in other fields. In other words, the 
three groups represent divisions of interests 
and not divisions of people, so that the same 
IRE members may be grouped in different 
ways for different fields. 

My feeling is that for groups A and C live 
technical meetings are at least as necessary 
and as effective as printed articles. How- 
ever, while technical meetings for group A 
should emphasize advanced research and 
allow ample time for discussion, the meet- 
ings for group C should consist primarily of 
up-to-date review and tutorial papers pre- 
sented by appropriate experts. Professional 


50 IRE TRANSACTIONS—INFORMATION THEORY 


Group symposia are eminently suited to 
meet the needs of group A. In this respect I 
believe that the organization of the 1954 
Symposium on Information Theory was a 
step in the right direction and that the 
effectiveness of future similar meetings will 
increase, once the audience becomes accus- 
tomed to carefully reading the printed 
papers ahead of time. On the other hand, 
the sessions at the National Convention 
should be devoted primarily to meet the 
needs of group C through the presentation 
of review and tutorial papers, and of original 
papers of broad significance. This suggestion 
appears to be in line with the present edi- 
torial policy of devoting the PROCEEDINGS 
or THE IRE to contributions of interest to a 
broad audience and to invited tutorial 
papers. Some of the review papers could well 
be reports on the latest developments that 
have emerged at Professional Group Sym- 
posia or other important technical meetings. 
It should be noted in this connection that 
the National Convention is just the occa- 
sion in which IRE members have the oppor- 
tunity of keeping themselves informed about 
a number of fields without any undue ex- 
penditure of time. Personally, I feel that in 
addition to the pleasure of meeting old 
friends, it is a broad up-to-date education 
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The Information Theory and Modulation 
Systems Committee, under the Chairman- 
ship of J. C. Kreer, Jr., is now working on 
the very difficult problem of definitions in 
the field of information theory. Among the 
definitions now being considered are: 


Bit: A unit of information content or capac- 
ity, equal to the information content in a 
binary decision between equally probable 
states. 


that I would like to obtain from the Na- 
tional Convention. 

The needs of groups A and C can thus be 
met without much difficulty. On the other 
hand, the organization of technical meet- 
ings designed to meet the special needs of 
group B presents serious problems. Any 
meeting dealing with developmental work 
is bound to be attended by such a large 
number of people that worthwhile discussion 
from the audience is precluded for all prac- 
tical purposes. Even the formal presenta- 
tion of original technical papers constitutes 
a very serious problem, because very often 
the importance of developmental papers 
resides in details, the significance of which 
can hardly be conveyed in a short time to an 
audience unfamiliar with the background 
of the specific problem. Perhaps review 
papers covering systematically recent work 
in each problem of interest are still a reason- 
ably good answer to the needs of group B. 
In any case, it seems to me that the needs 
of group B can be met more realistically by 
the quick distribution of printed literature 
than by live technical meetings. 

The fact of the matter seems to be that 
the needs of groups A and C are still fairly 
similar to those felt back in the past by 
members of learned societies, and therefore 


Channel: A transmission path and associated 
terminal equipment capable of receiving 
signals at one point and delivering re- 
lated signals at another point. 


Code: A set of transformation rules to be ap- 
plied to messages or signals. 


Code Character: The signal representation of 
a discrete value or symbol in a message. 


Code Element: One of a finite set of parts 
from which code characters may be con- 
structed. 


Hartly: A unit of information content or 
capacity, equal to the information con- 
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can still be fulfilled by meetings and publi- 
cations of the traditional form. The exis- 
tence of very large groups of the B type, on 
the contrary, is a new phenomenon resulting 
from the tremendous expansion of the elec- 
tronics industry in the last decade. The 
needs of these groups present new problems 
on which little experience is yet available, 
and which are often made more difficult by 
failure to recognize their nature. 

The problem of how the Institute can 
best keep serving the technical needs of its 
fast growing membership deserves careful 
thought on the part of all concerned. Per- 
haps the classification of interests used 
above is not quite appropriate. Yet, I feel 
that full recognition must be given to the— 
wide range of interests—research, develop- 
ment, management, etc.—required in the. 
modern electronic industry, and to the obli- 
gation of serving all such interests with 
equal professional and intellectual stand- 
ards, within the framework of the Institute 
of Radio Engineers. 


R. M. Fano 

Associate Professor of 
Electrical Communications 

Massachusetts Institute of Technology | 

Cambridge, Mass. 


tent in a decimal decision between ten 
equally probable states. 


Information Content (of a message from a 
source): The negative logarithm of the 
probability that that particular message 
will be emitted by the source. 


Signal: The physical embodiment of a mes- 
sage. 


The Committee is anxious to receive any 
comments, variations, or definitions that 
IRE members may have as soon as possible. 
Comments sent to the Professional Group 
on Information Theory will be forwarded to 
the Committee. 
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Authors are requested to submit editorial correspondence or technical manu- 
scripts to the Publications Chairman for possible publication in the PGIT Trans- 
AcTIONS. Papers submitted should include a statement as to whether the material 
has been copyrighted, previously published, or accepted for publication elsewhere. 


Papers should be written concisely, keeping to a minimum all introductory 
and historical.material. It is seldom necessary to reproduce in their entirety previ- 
ously published derivations, where a statement of results, with adequate references, 
will suffice. ; 


To expedite reviewing procedures, it is requested that authors submit the 
~ original and two legible copies of alk written and illustrative material. The manu- 
script should be double-spaced, and the illustrations drawn in india ink on drawing 
* paper or drafting cloth. Each paper should include a carefully written abstract of 
~ not more than 200 words. Upon acceptance, papers should be prepared for publica- 
tion in a manner similar to those intended for the ProcrEpINGs oF THE IRE. 
Further instructions may be obtained from the Publications Chairman. Material 
not accepted for publication will be returned. 


IRE TRANSACTIONS ON INFORMATION THEORY is published three times a year, 
in March, September, and December. A minimum of one month must be allowed 
for review and correction of all accepted manuscripts. A period of approximately 
two months additional is required for the mechanical phases of publication and 
printing. Therefore, all manuscripts must be submitted three months prior to the 
respective publication dates. There will be no June issue, since the IRE CoNvENTION 
ReEcorp is published at that time, and a bound collection of Information Theory 
papers previously delivered at the 1955 Convention will be mailed gratis to all 
PGIT members. 


All technical manuscripts and editorial correspondence should be addressed to 
Laurin G. Fischer, Federal Telecommunications Labs., 492 River Road, Nutley, 
N. J. Local Chapter activities and announcements, as well as other nontechnical 
news items, should be addressed to Nathan Marchand, Marchand Electronic Labs., 
255 Mill Street, Byram, Conn. 


& 


